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ABSTRACT 


A method for predicting Replacement Air Wing Carrier 
Qualification grades is examined. ‘The data were supplied 
by Fighter Squadron One-Twenty-One, and subjected to multiple 
regression analyses in search of imoortant variables that 
may be used in the prediction. Sucn have been identified 
and applied to the data. The results are extremely 
encouraging and a follow on study avplied to a broader data 
base is suggested. Applicability and some economic factors 


are discussed. 
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I. INTRODUCTION 


Poctorveally, ene Gistinguishing characteristic of a 
Naval Aviator has been his ability to land a high performance 
Peeecrart On the deck Of an alrcraft carrier at sea. This is 
not only a proud trademark, but is Sonia by many to be 
the real mark of success and professionalism for pilots, 
for while many individuals have learned to fly quite 
competently, relatively few ever master the ipioet each ae Of 
carrier landing techniques. In large measure, then, if we 
are able to predict a pilots chance of success in mastering 
the carrier landing game, we will have predicted to a great 
extent his value to a squadron, and to the Navy. There can 
be no doubt, and we will examine some statistics to show 
this, if we are able to predict a lack of success at the ship, 
we can Save vast amounts of money, time, and most important 
lives. 

The particular problem addressed here came to light 
during a tour as the senior landing signal officer for the 
Pacific Fleet F-4 replacement training squadron, Fighter 
Squadron One-Twenty-One, (VF-121). Due to stringent limita- 
Elons von Gime, ailrerart, and Carrier availability it became 
mandatory to try to pinpoint those individuals who would 
Witimately Have a Significant amount of difficulty during 


CAbrele CUeiitications very Carly in their training in order 





to devote extra time and attention where it was available, 

in hopes of ensuring their ultimate success. It was felt, 
aede by OnpenieetastesOl expervence, that instrument training 
grades in the Advanced Training Command, and in the Replace- 
ment Air Wing's instrument training Squadron, were the 
strongest possible indicators of potential success or failure, 
but these predictions were based only on an intuitive "feel" 
fer che Situation. This project was started in order to 
attempt to provide some statistical support for these theories, 
or to debunk them thoroughly should that prove the case. [In 
the event they proved successful, the object was to provide 
some formulation which would permit accurate predictions of 
CorEler Qualification performance, atilizing any factors 


which might be available in normal circumstances. 





iene wOr eine PROBLEM AND APPROACHES TAKEN 


Snecompletionsor the Flight Training Command, the newly 
designated aviator (nugget) proceeds to one of the cones 
Replacement Air Wings, accompanied by a relatively small 
and incomplete sample of his performance during training. 
mee tollowing specific scores, or crades, aac Geeenaeat 

1. Basic Training Command Standard Score 
2. Acvancea Training Commend Standard Score 
Bae Grounes se CiOol Stancarad Score 
4. Stage Grades from Advanced Training to include: 
awe femiltarrzat von 
b. Basic Instruments 
c. Instrument Navigation 
ad. Advanced Familiarization 
@. Formation and Tactics 
Eweenboghit Pamillarization 
g. Operational Navigation 
hee er —-tO-Ground 
Pee rackies 
Fo habia ele ey \alg 
Keecarrier Qualifications 

It must be recognized that a sizeable investment has been 

Made wueenesindividual pilot Eo this point, a “subject of 


comsicgenaole Study im 1tself., In a paper written for the 





Naval Personnel Research and Development Laboratory in May 
of 1970, Joseph Taylor and Simon Arzigian break down the cost 
memean F—-4 pilot as follows: 


Naval Aviation Schools Command See, 4000 


Primary / Basic Training 227 os ..00 
Advanced Training SiZeye4 s.00 
Total S129 7183200 


The same reference contains Similar figures for pilots trained 
in other aircraft types, although these figures are basically 
constant across the board. The aviator has received 14 to 

¥8 months of intensive training to prepare him to step into 
pmconNbat dgircraft. In the next 90 to 180 days, the first 

year and a half he has spent in the Naval Aviation business 
will be made to look like grammar school, as he compares it 

to the postgraduate school of Naval Aviation, the “RAG". Here 
he will receive an intensive course designed to make him a 
combat ready fleet pilot, a course that culminates in Carrier 
GOmalitication in his particular aircraft, and which will 

cost the Navy another $163,776.00, in the case of the F-4 
pilot. It should be apparent from these figures, that the 
ability to predict carrier qualification performance early 
leche game would pay off, not only in pinpointing specific 
individuals as being marginal, but can, if done early enough 
and with high enough accuracy, Save a substantial sum of 
money by merely eliminating those individuals who are doomed 


Pour atiure Lromoche Start. 





There have been numerous attempts at finding correlations 
which might prove useful, perhaps one of the more interesting 
meecOntained in a paper written for the Naval Aerospace 
Medical Research Laboratory, Pensacola, Florida, by Lieutenant 
George M. Fickus Jr. Medical Service Corps, United States 
Nawal Reserve. Lt. Rickus reported on a survey of flight 
surgeons conducted over a two year period. The flight 
surgeons were asked to identify and classify unsatisfactory 
aviators. The classifications were delineated and the 
flight surgeons identified 144 such aviators. The following 
breakdown resulted: 

24 Men other pilots refused to fly with 

43 Turned in their wings 

32 Had their wings removed by board action 
22 Were transferred administratively 

23. Were given non flying duties 

A cone! group of 146 aviators were also identified, 
these men were characterized as the "better" aviators in 
their respective squadrons. (Apparently Lt. Rickus had 
mlcinined On USING nOn-parametric methods, he does not so state, 
however, and the reason for this particular control group 
is never made clear in his paper.) 

‘Lt. Rickus states, "A comparison of the available 
training statistics on the two groups indicated significant 


@irrerencesein three areas: 


l. Peer rating 

2. §8asic Carrer Oualification 

3. Advanced Flight Grade " 
emis Signiiicant that two or the three areas found signifi- 
cant in this study are areas which are not forwarded to the 
Replacement Air Wing unless specifically requested, these 
are the Peer rating, and BaSic Carrier Qualification scores. 

The table which follows is presented as part of Lt. 

Rickus report. It provides a breakdown and cumulative 
frequency distribution of the 99 aviators from the above 
Soup £LOr whom full training statistics exist, Compared to 
873 so called "Satisfactory" aviatcrs who had comparable 
Peaerstics available. 


PREQUENCY DISTRIBUTION OF BASIC CARRIER QUAL. GRADES 


Satisfactory Unsatisfactory — 

Grade Interval N = 873 N = 99 
Freq. Cumulative Freq. Cumulative 

Pemeewcade Percentage | 
Less than 2.70 9 1 5 5 
Pom 2. 15 i 6 
ee O-2. 80 a 10 
geol-2.85 36 > 10 20 
eeco-2 .90 36 9 8 28 
preol=2.95 81 Je 14 42 
eo -3.00 81 28 30 73 
Ol 3.05 ao 72 KG 83 
Be06-3.10 J26 87 5 88 
Sei-=3.15 ne a5 6 94 
3.16 or greater 43 100 6 100 








Although the explanation of the table as it is contained in 
Meee texus paper is incomplete at best it appears to point 
to the following conclusions: 

imeby Cstablrshing a Basic Carrier Qualification grade 

of 2.80 it will be possible to eliminate 10% of the 

unsatisfactory aviators, while causing an attrition of 
only 1% among the satisfactory group. 

2. By increasing the cutoff score to 2.90 we eliminate 

28% of the unsatisfactory aviators, while forcing a 9% 

loss in the satisfactory are 

DemretUckiCoT Increase to 2.95 Gauses an atiErition of 

42% of the unsatisfactory group, while losing only 19% 

©tethe Satisfactory group. 

The Eerm cutoff refers to simply terminating training for 
those individuals who fall below that grade point average for 
the stage involved. 

Unfortunately Lt. Rickus has not identified his groups 
well enough to allow us to accept these conclusions at face 
Value, in fact he presents no conclusions at all, merely 
offering the table as shown and allowing the reader to draw 
his own conclusions. It would be nice to be able to assume 
that the sample is a complete cross section of the fleet 
during the time that the study was undertaken, and that the 
conclusions we have drawn therefore hold true across the 
board in Naval Aviation. Since that issue is not addressed 
throughout the paper, further research is called for. If that 


assumption holds true the implication would be obvious, 
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Berticulatly in the light of the cost figures quoted earlier. 
The optimum policy would simply be, maintain a very high 
fear 7e CULOIL Point for Basic Carrier Qualifications, and 
eulow Only limited access to the Advanced Training Command. 
mie end product should improve markedly, and the cost per copy 
should decrease. Even without further research this appears 
MOmecean acceptable method for obtaining a high quality 
Mecauct, the decision being one of understanding and accepting 
fee tradeort. 

Another study, under the auspices of the same group, 
dated 2 March 1970, indicates a strong relationship between 
basic carrier qualification, and Advanced Training in the jet 
Semmunity, but little success in correlating Basic CQ with 
performance in the Combat Replacement Air Wing. The study 
indicates a very small correlation between Basic CQ and 
advanced multi-engine training, but it is felt that the results 
of the first study referenced, if they may be accepted, and 
ine small additional cost of Basic CQ training for this 
category of pilots, provides a strong argument for allowing 
Bee taxed wing pilots to complete Basic CQ training, and use 
the results as a very definite attrition point. 

It will be our purpose now to branch and look specifically 
Beecne Jet community, and more particularly the F=4 pilots 


to see if some further predictors may be determined. 


lial 


Pelee ence MENTAL PROCEDURE 


As has been stated, it 1S apparent that we should attempt 
to identify, as early as possible, those individuals who 
will fall in a marginal or unsatisfactory category during 
shipboard operations. Clearly, if the unsatisfactory 
individual can be identified, there is but one course of 
action, immediate forced attrition. In the case of tiles 
marginal individual some special attention may be sufficient 
to save the situation, and the money already invested. 
Towards this end a study of the "nugget" Naval Aviators who 
completed the West Coast F-4 Combat Replacement Air Wing, 
during a two year period, has been completed. The purpose 
of the study was to determine whether it was possible to 
predict performance on Carrier Qualifications within 
meeeptable limits of accuracy. 

Whe Most £rustrating part of the study was the collection 
of data. The proposed method of attack required a complete 
stage breakdown from the commencement of the Basic Training 
Command on through to eventually joining a fleet squadron. 

In addition it would be necessary for the sample size to be 
large enough to be satistically significant. It rapidly 
became apparent that, while various places maintain summaries 
and frequently standard scores, no one, easily accessible, 
location had the breakdown required. The single most helpful 


item was the fact that VF-121 at the Naval Air Station, 
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Miramar, California, maintained a file of training jackets 


covering several years, containing not only a stage by stage 


Peeakdown Of training in the F-4, but, in addition, containing 


grades from the instrument refresher squadron, VF-126, and 


the list of grades alluded to earlier in this paper. 


It is appropriate to comment at this point that an 


examination of grading statistics from the Basic and Advanced 


Training Commands show a very small standard deviation from 


the mean score for each 


Standardization in both 


For the sample examined 


Standard deviations are 


Te 
ie. 


stage, indicating a high degree of 
instructional and grading techniques. 


the mean stage grades and associated 


given: 

Mean SEC DEW. 
Familiarization a20G1 17 OF 07527 
Basic Instruments 5. OSS 2 Og 5305 
Instrument Nav. 6.05660 0.04591 
Advanced Fam. BU 2619 Oe0'3 597) 
Mohaite ( dieleiae 3.06449 0.04441 
Night Fam. Se eG0g5 0.04801 
Operational Nav. 2.08233 Oe,0572 3 
Alr=terGcround SOM Gz 05,0 7317 
Taceues 3209429 0205799 
AlYyS=e@-Air Be OUsLIG 020502310 
C.Q. 2.01055 0706623 


Only those individuals who had never been carrier 


qualified in a fleet operational aircraft were considered in 


the sample. 


Complete statistics were available on 158 such 


ne 





individuals, of these 8 were known to have had major difficulty 
Yanding aboard ship after joining their squadron, and had 
Peeweremoved from Elying Status either voluntarily, or through 
evaluation board action. It is interesting that no aviators 
Pemaeeogitvyen evaluation boards for performance during Carrier 
Qualifications during this period, although several had beer 
recommended by the Landing Signal Officers involved. Of the 

8 individuals noted above, all had Carrier Qualification 
grades between 2.00 and 2.58 while in VF-121. Two other 
individuals fell within that same grade category, but in 

both cases they had gone on to do an acceptable job in the 
Squadron. The mean score for Carrier Qualification in VF-121 
for the entire sample was 2.81868, and the standard deviation 
wee 0.25700. 

Om the baSis of the records noted, the fact that 2.58 is 
almost exactly one standard deviation below the mean, and a 
certain amount of intuitive appeal, it was decided to try 
eseaplishagq 2.58 aS a cutofE score. Any predicted Carrier 
Qualification grade below 2.58 constitutes justification for 
Gisconcinuinge training. To lend statistical credence to the 
hvpethesis that 2.58 1S a reasonable cutoff point, a discrim- 
inant analysis was performed using the UCLA Health Sciences 
Computing Facility BIMED program. The F test performed on 
the data reveals a significant difference in the two groups 
Wienmeusing 2.50 aS the Cutoff peimt, Significant to any level 


tabled in standard references, or at least the .999 level. 
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Two standard IBM package programs were then applied to 
the data, STEPR, and REGRE. STEPR takes the individual 
independent variables in a regression analysis, determines 
which order they should enter in, and enters them. REGRE 
takes the variables and enters them in the order instructed 
beeetne user. The use of the computer at this point was 
obvious and necessary. While the computations are not at all 
G@itticult, they are extremely time consuming when the number 
of possible combinations of indeperdent variables is taken 
into consideration. (15 independent variables were checked} 
The computer permitted arapid survey of all variables to 
determine correlation with the dependent variable (C.Q. scoxe), 
and was able to perform rapidly the computations using many 
Bapterent combinations of variables in a search for the 
strongest possible predictors. 

ASesuSPeGrea mene MOSt POWemnul predictor was, One Gia enc 
instrument grades. It was not, however, one of the Advanced 
Training Command grades as had been expected. The Instrument 
Refresher Squadron, VF-126, had the strongest positive 
correlation (.556). There are a number of potential explana- 
tions for this, none of which have anything more than 
intuition to support them as theories. The most appealing is, 
the newly designated aviator has been out of the cockpit for 
Six to Eight ae while he has been on leave and in transit 
Grom his last duty station. When he climbs into the plane 
for the 10 flights in the Instrument Refresher syllabus he is 


not as sharp as he might be, and in fact often has a good deal 


AS) 





of apprehension about it. The tension generated, coupled 
with the fact that he is in a foreign atmosphere for the 
first time in weeks tends to simulate quite well the mental 
and physical strains he will encounter at the ship when he 
goes ue £Or Ene rirst time. In addition the instrument 
skills he will be called on to use are exactly those he will 
be called upon to use a great deal in shipboard operations. 
Although 2 1s not possible to quote any authority as 
justification for these conclusions, they seem further 
Supported when the second and third best predictors are found 


to be the! Fam stage in the Advanced Training Command), and the 


ee ee 
Fam stage in the fleet aircraft itself. In each case similar 
tensions are readily apparent. It would be possible to go 


down the list of correlations, but that hardly seems bene- 
ficial in this case, we should note in passing, however, that 
there are two stages with pronounced negative correlation 
Wie carrier Qualifications, they are Formation and Tactics, 
eaiaeALy—-LO-ALY . 

Once the correlation coefficients had been obtained, 
linear regression methods were employed in an attempt to 
predict performance not only reliably, but as early as possible 
as well. Once again the computer became an invaluable tool, 
for the method employed had to be largely trial and error. It 
was found that the highest accuracy could be obtained by 
permitting the pilot to complete the Familiarization, and 
Instrument Training stages within the F-4 squadron, however at 


that point a sizeable amount of ground school training, and 
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some 10-15 hours of flight time has been devoted to the 
mratvidual pilot, a significant amount of the training funds 
have already been committed. An att.empt was therefore made 
to back off and make the prediction as early as possible 
mmewout a Significant reduction in accuracy. The Timit, with 
the data currently available, seems to be the completion of 


the Instrument Refresher Squadron, the following formula is 


appropriate: 
Predicted C.Q. = -1.64886 + 0.738344 (a) + 0.71096 (b) 
where (a) = Instrument Refresher Squadron Grade 
(b) = Familiarization Stage grade from Advanced 


Training Command. 
The standard error of estimate achieved using this formula 
is 0.191, and the accuracy of the predictions has some 


interesting characteristics. 
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IVs) RESULTS AND CONCLUSIONS 


imo reaSeuCtGdstie fOmmula predict that any individual 

who was ultimately successful in the fleet would fall below 
Eheecutoff point of 2.58, that was true even in the case of 
the two pilots who had low Carrier Qualification grades, but. 
had done an acceptable job in a fleet squadron. All of the 
8 individuals who had eventually proven unacceptable were 
pinpointed by the formula. Closer inspection of the 
individual scores involved revealed that the two who were not 
predicted to have problems had mean scores in the two 
Significant areas as shown: 

a. Instrument Refresher OE ti8 

bo Hem. Stage 3200 
On the other hand the 8 pinpointed by the formula had: 

a. Instrument Refresher Zo 

bewelanme otage 3. 06 
Note that the Instrument Refresher grade was higher for the 
two aviators not identified by the prediction equation, whereas 
the Fam. stage grade was lower. This might appear as a 
discrepancy because both of these variables are positively 
Correlated with the Carrier Qualification forecast. Closer 
examination reveals that the above two Instrument Refresher 
grades are more than four standard deviations apart (n.b. 


standard deviation for this stage is .13363) whereas the Fam. 
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stage grades differ by less than one standard deviation (Fam. 
Plage Standard deviation 1s .0/527). Thus, in the particular 
cases examined the spread in the one variable overwhelms the 
pac (reversed) spread in the other. This phenomena is not 
at aia unusual in the application of regression analysis. 

Claims of 100% accuracy are at best unwise, and often 
refutable, but the evidence does seem to support a much more 
fetailed analysis of a broader group of data, particularly 
Since all estimates generated by using this formula erred on 
the high, or safe, side. There Sd, Ana areas which will 
require financial and administrative support not obtainable 
at this time. 

ire hCculesoebepOreea On abe only valid) im thescase/ or 
Fighter Squadron One-Twenty-One, and may not be applied across 
the board without further investigation. Additionally the 
results available to this point would have value only to the 
Commanding Officer of the Fleet Replacement Training Squadrons, 
and not to the Bureau of Naval Personnel. These problem areas 
may both be dealt with simply by providing adequate support, 
in the form of the required grading statistics for the most 
part, to enable the area to be more thoroughly researched. 
This must include a complete stage breakdown from the time the 
subjects enter the Training Command, until joining a fleet 
squadron, and must constitute a statistically significant 
jcqmplesiOrveach Of the Separate aviation communities by air- 


Ceant type. lt is felt that this paper clearly demonstrates 


Ai) 





SecUlooctanite lal weetentlal EOE Signiticant long term gains, 


beth financially, and in the quality of fleet aviators. 
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